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THhe opinions expressed in this paper are those of the authors 
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I INCENTIVE POLICIES AS TOOLS TO FURTHER STATE AND 

FEDERAL JTP A GOALS 



A. INTRODUCTION 

The purpose of this technical assistance guide is to help state 
policymakers to develop performance-standard incentive policies that 
are effective tools in accomplishing state and federal goals for JTFA 
programs. Incentive funds are valued by SDAs because they are 
tangible rewards for accomplishing program goals, can be used more 
flexibly than other funds, and can help cushion declining allocations in 
some SDAs. Incentive policies, therefore, are a powerful mechanism 
for states to guide the JTPA program. 

To be effective, however, incentive policies must clearly reward 
the attainment of the state's goals. Unfortunately, the incentive 
systems in many states have not been designed to accomplish specific 
purposes but instead have evolved over time as technical responses to 
legislative or regulatory requirements rather than as a means to 
promote state goals. This guide is intended to help states reexamine 
and, if necessary, revise their incentive policies to accomplish 
intended goals and avoid unintended results. 

Developing incentive award policies involves a four-step process; 

■ Establish clear goals to be accomplished by the state's JTPA 
program. 

■ Select performance outcomes that are consistent with those goals 
and that measure the attainment of the goals as closely as possible. 

■ Set realistic expectations for attainment of performance outcomes. 

■ Develop incentive award policies that reinforce the attainment of 
the state's goals. 

The remainder of this chapter summarizes the federal goals for 
JTPA and highlights policy mechanisms available to states to guide 
the JTPA program. It concludes with a summary of principles that 
should be followed in designing effective incentive policies. 
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B. FEDERAL AND STATE PROGRAM GOALS AND POLICIES 



Federal goals for the JTPA program include the promotion of 
services that: 

■ Improve the long-term employability of JTPA participants. 

■ Result in the placement of program terminees in quality jobs. 

■ Address the acquisition of basic skills by individuals with basic 
skills deficiencies. 

■ Take advantage of effective coordination of services and funding 
streams at the state and local levels. 

■ Target the needs of individuals with serious employment barriers. 

These goals ?je reflected in the federal measures chosen for 
PY 90. The adult and welfare follow-up employment measures more 
adequately reflect the program's ability to improve participants' long- 
term employability than did the termination-based employment 
measures. The adult and welfare measures of weekly earnings at 
follow-up reflect the attainment of quality jobs. The youth 
employability enhancement measure emphasizes that improving the 
long-term employability of youth involves more than finding them 
immediate employment and that addressing basic skills deficiencies is 
a valuable outcome for JTPA. The youth entered-employment rate 
will be applied to youths for whom employment is the immediate goal. 

Further, beginning in PY 90, cost measures are not included as 
federal measures and states are precluded from awarding incentives 
for exceeding cost measures. As described in Chapter II, cost 
standards have substantial unintended effects of reducing services to 
individuals with serious employment barriers and reducing the 
provision of basic skills services. 

Thus, the federal performance standards follow clearly from goals 
that DOL has established for the JTPA system. The federal 
performance standards, however, were never intended to be the only 
source of policy guidance for the JTPA system. As shaped by the 
framers of the JTPA legislation, the JTPA system provides substantial 
discretion to states and SDAs to develop policy goals for JTPA client 
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targeting and service strategies in keeping with local concerns and 
priorities. 

The JTPA legislation invites states to be active partners with the 
federal government in shaping JTPA policies to further a variety of 
performance, client, and service goals. States have the ability to 
provide policy guidance and leadership to SDAs on these issues 
through a variety of mechanisms, including: 

■ The development of state policies about priority clients in the 
Governor's Coordination and Special Services Plan. 

■ The shaping of the state performance-standard system, including 
the choice of additional state standards and the design of incentive 
policies. 

■ The design and implementation of technical assistance efforts to 
improve program effectiveness and promote innovative services. 

■ The design of data collection systems to monitor program 
achievements. 

« The establishment of policies and mechanisms to promote 
coordination between JTPA and other agencies. 

Thus, performance standards are only one mechanism for 
communicating and reinforcing state goals for the JTPA program. It 
is important, therefore, for other state policies to work together with 
performance-standard policies to further state goals. 

C. SUMMARY OF DESIGN PRINCIPLES 

As discussed above, the first principle in designing incentive 
policies is that the policies should be explicitly chosen to reinforce 
state goals for the JTPA program. In doing so, however, several other 
factors should be considered. It is important to recognize that 
tradeoffs among competing goals may be necessary. For example, the 
goals of maximizing local autonomy in program design, encouraging 
service to the hard-to-serve, and maximizing performance on outcome 
measures are likely to conflict in designing an incentive policy. The 
incentive policy, therefore, should clearly reflect the state's priorities 
when tradeoffs are required. 
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A related principle is that the incentive policy should not only 
accomplish its intended effects but also avoid unintended effects. The 
next section of this guide, therefore, highlights both the intended and 
unintended effects of various policy options. 

Another design principle that is very important for effective 
communication of state policy goals is simplicity. This goal can be 
very difficult to achieve in practice because many different design 
elements need to be taken into account in developing incentive award 
policies. It should be kept in mind, however, that the ultimate goal is 
to influence SDAs' behavior. If SDA staff do not understand an 
incentive award formula, they cannot respond to it appropriately in 
designing and operating their JTPA programs. 

The remainder of this guide presents information to help states 
design incentive policies that promote state goals and priorities, avoid 
unintended effects, and are as simple as possible. Chapter II presents 
a detailed step-by-step guide to the elements in states' incentive 
policies. It presents options that states have chosen to emphasize 
different outcomes, to determine when SDAs quality for incentive 
awards, and to calculate the amount of incentive payments based on 
SDAs' performance relative to their standards. For each element, we 
present the intention of the policy, the number of states that used the 
policy in PY 88, a specific example of the policy, and the effects of 
that policy on performance achieved, clients served, and services 
provided. 

Chapter III presents case studies of five state policies. These 
policies were chosen not necessarily as models for other states to 
adopt but rather as examples of coherent policies in which all 
elements work together to accomplish clearly established state goals. 
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II ELEMENTS IN STATE INCENTIVE AWARD POLICIES 



State incentive policies are composed of four basic policy 
elements: the emphasis placed on different outcomes, how SDAs 
qualify for incentive awards, how incentive awards are calculated, and 
what conditions are placed on the use of incentive funds. All four of 
these basic elements should be designed to work together to further 
state goals for JTPA. 

This chapter presents a step-by-step guide to developing incentive 
policies that address the following key questions within each of the 
four basic elements of incentive policies: 

Emphasizing different outcomes*, 

a Do the federal standards adequately reflect state goals for JTPA 
or should additional state standards be added? 

■ Are all outcomes equally important or should key outcomes be 
emphasized over other outcomes? 

Qualifying for incentive awards: 

■ When is an SDA's performance on an individual standard 
sufficiently high to have met that standard? To have exceeded 
that standard? 

■ When is an SDA's performance on all standards sufficiently high 
to qualify for incentive awards? 

Calculating incentive awards: 

■ Should a separate pool for each standard be established or should 
a composite measure of performance be used? 

■ How much should SDAs receive for marginally exceeding their 
standards? 

■ How should performance beyond the standard be rewarded? 

■ Should there be a cap on rewarded performance beyond which no 
additional incentive awards are received? 
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■ Should awards be determined competitively so that how much one 
SDA receives depends on how well other SDAs in the state 
performed? 

■ Should the amount of incentive awards depend on the size of the 
SDA 9 

Conditions placed on incentive funds: 

■ Should incentive funds be exempt from performance standards? 

■ Should the state require incentive funds to be used for specific 
purposes? 

For each of these aspects of incentive policies, this guide presents: 

■ The intention of the policy. 

■ The number of states that chose the policy in PY 88. 

■ An example of the policy. 

■ The effect of the policy, both intended and unintended, on clients 
served, services provided, and performance achieved.* 

A. EMPHASIZING DIFFERENT OUTCOMES 

States can affect the direction of the JTPA program by 
emphasizing the outcomes for which SDAs will be rewarded in their 
incentive policies. The first step is to decide whether the federal 
standards adequately reflect the state goals for JTPA or whether the 
state wants to establish additional performance measures. The next 
step is to decide whether all the chosen outcomes are equally 



• Information about the intended and unintended effects of state policies was obtained from 
"JTPA Performance Standards: Effects on Clients, Services and Costs" (NCEP, 19S8) and * 
subsequent study, Th« Effects of JTPA Performance Standards on Performance" (NCEP, 
forthcoming). Information about states' intentions for their policies was obtained from 
discussions with state staff as part of the NCEP studies and from The Study of Exempted 
6% Projects" (SRI, forthcoming). Information about states' incentive policies was obtained 
from the Governor's Coordination and Special Services Plans for PY 88. 
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important or whether key outcomes should be emphasized over other 
measures. This section describes these two steps: 

■ Choosing additional state performance standards to be used in 
awarding incentives. 

■ Emphasizing performance on key standards in calculating 
incentives. 



1. Choosing Additional State Performance Standards 
Options for State Standards 

States have chosen four types of additional standards: 

A. Service levels to hard-to-serve groups 

B. Outcomes for hard-to-serve groups 

C. Additional outcomes 

D. Other program goals. 

Option A: Standards for Service Levels to Hard-to-Serve 
Groups 

Intentions of Policy. States have adopted additional standards for 
service to hard-to-serve groups to increase the number of at-risk 
individuals served by JTPA. 

Past Experience. In PY 88, 14 states set standards for service to 
hard-to-serve groups. Eight states set separate standards for the 
proportion of terminees that were in specific groups, such as dropouts, 
welfare recipients, minorities, women, and handicapped. In a few 
cases, service to the hard-to-serve was not identified explicitly as a 
state standard, but the level of service to specify target groups was 
incorporated into the calculation of incentive payments. Eight states 
set a standard for the proportion of terminees that met a broader 
definition of hard to serve. 

Example of Policy. Kansas set separate standards for the level of 
service to AFDC recipients, females, and minorities for PY 88/89. 
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The service level standards for AFDC recipients and minorities were 
adjusted to reflect the incidence of the target group in an SDA's 
eligible population, and the standard for females was set at 57%. Half 
of incentive awards were given to SDAs that exceeded target levels for 
the specific hard-to-serve groups. SDAs had to exceed service levels 
for all three groups to be eligible for the incentive award for service to 
targeted groups. Additionally, the state assigned separate weights to 
each target group-AFDC recipients (50%), females (25%), and 
minorities (25%)~thus providing greater incentives for serving AFDC 
recipients. 

Option B: Standards for Outcomes for Hard-to-Serve 
Groups 

Intentions of Policy. States adopting outcome standards for 
specific groups wanted to ensure that SDAs achieved valued outcomes 
for hard-to-serve participants as well as other J TP A participants. 

Past Experience. In PY 88, 7 states set a standard for the level of 
outcomes attained for hard-to-serve participants. Most often, the 
outcomes were placement, although 3 states established positive 
termination rates and one set a wage at placement standard for hard- 
to-serve participants. 

Example of Policy. Arizona set two state outcome-oriented 
performance standards for hard-to-serve groups: a hard-to-serve 
positive-termination rate and an adult and youth AFDC entered- 
employment/retention rate. Specific hard-to-serve groups identified 
by the Governor included: 

Welfare recipients 
Handicapped individuals 
Offenders 

Individuals with limited English proficiency 
Migrant or seasonal farm workers 
Displaced homemakers 
Substance dependent individuals 
School dropouts. 
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To receive an award, the SDA's positive terminations (as defined 
by JTPA Administration) for these groups had to exceed 40% of all 
SDA positive terminations. 

Arizona's Adult and Youth AFDC Entered Employment/ 
Retention Rate was designed to reward SDAs that place and retain 
AFDC recipients in unsuhsidized employment for 90 days. The 
incentive award is determined by the extent the SDA's performance 
benchmark is exceeded. 



Option C: Standards for Additional Outcomes 

Intentions of Policy. States have chosen standards for additional 
outcomes for several reasons. In PY 88, several states chose more 
than eight federal standards, often so that SDAs would be held 
accountable both for the more familiar termination-based outcomes 
and for the new follow-up measures. Often this was seen as a 
transition strategy, although some states may wish to continue using 
termination-based standards in PY 90 as well. 

Other states applied adult outcome measures to youths as well 
because they felt they were valid outcomes for all JTPA participants. 
Another reason additional outcomes were chosen was to measure the 
quality of jobs obtained through JTPA, including measures for wage 
gains and placements in growing industries. 

Another option that states may wish to consider is adding a 
standard for intermediate outcomes for adults, such as the number of 
individuals with basic skills deficiencies that attained additional basic 
skills or a GED. The intention of this standard would be to 
encourage SDAs to provide basic skills training. 

Although not adopted for PY 90, consideration was given at the 
federal level to establishing a "value-added" placement rate standard 
that would count the number of adults who attained basic skills or job- 
specific skills and who entered employment. The intention of such a 
standard would be to discourage less intensive services, such as job 
search assistance only, and encourage more intensive services that 
would have a potentially greater effect on long-term employability. 
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Pr«t Experience. In PY 88, 10 states set standards for additional 
outcomes. Five designated as state standards additional standards 
chosen from the federal menu of measures, and 5 set standards for 
other types of outcomes. 

Example of Policy. Washington State established a youth retention 
rate standard. The standard was determined by dividing the number 
of youths employed at both follow-up and termination by the number 
of youths who were employed at termination. Washington developed 
a statewide regression model to adjust its youth retention rate 
standard. 

Option D: Standards for Other Program Goals 

Intentions of Policy. Some states have incorporated other goals 
into state standards because incentive payments are a useful tool for 
rewarding the achievement of these goals. 

Past Experience. In PY 88, 5 states set standards for other program 
goals, including expenditure rates and coordination with other 
programs. In addition, although expenditure rates were not explicitly 
identified as a state standard, 15 states incorporated the expenditure 
rate into the calculation of incentive awards. 

Example of Policy. New Mexico set two Governor's standards: an 
85% planned expenditure rate and an 85% planned enrollment rate. 
Twenty percent of available incentive awards are distributed to SDAs 
that exceed these standards. 



Effects of State Performance Standards 

State standards can affect clients served, the services provided, and 
program performance. As summarized in Exhibit II- 1, standards for 
service levels for specific target groups do have the intended effects of 
increasing service to those groups. Performance standards for service 
to welfare recipients also increase the number of participants 
receiving basic skills training. Standards for service to welfare 
recipients tend to decrease performance on the wage at placement 
and earnings at follow-up measures. 
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Exhibit II- 1 



SUMMARY OF EFFECTS OF ADOPTING STATE STANDARDS 
FOR SERVICE TO WELFARE RECIPIENTS AND DROPOUTS 

State Standards for Service to Welfare Recipients 



adult welfare recipients served standards for wage at placement 

and weekly earnings at follow-up 

Increases the percentage of 
adults and youths receiving basic 
skills training 



Intended Effects 



Unintended Effects 



Increases the percentage of 



Decreases performance on 



State Standards for Service to Dropouts 
Intended Effects Unintended Effects 



Increases the percentage of 
adult dropouts served 



Increases the percentage of 
adults ani youths receiving job 
search assistance 



Reduces performance on 
employment measures 
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Standards for service levels to dropouts increase enrollment of 
dropouts but do not have the intended effect of increasing the 
provision of basic skills services. Instead, standards for service to 
dropouts increase the number of individuals receiving job search 
assistance from JTPA. In site visits to SDAs, it was found that some 
SDAs provide dropouts either OJT or only job search assistance 
because their classroom training service providers require enrollees to 
have a high school diploma State standards for service to dropouts, 
therefore, are not sufficient to ensure that dropouts receive quality 
services. Standards for service levels to dropouts tend to reduce 
performance on employment-related outcomes. 

The effects of outcomes for hard-to-serve groups cannot be 
separated from the effects of service levels because most states that 
set outcome standards for the hard-to-serve also establish service level 
standards as well. Too few states adopted specific measures to 
determine their effects on other outcomes. However, in general, 
SDAs tend to respond to additional standards. 

Although additional state standards are powerful policy tools that 
states can use to affect the direction of JTPA, another unintended 
effect can occur if too many standards are added or if the goals of the 
standards become confusing or conflicting. Findings from site visits 
indicate that some policies were trying to accomplish too much, and 
SDAs could not plan to achieve all the outcomes at once. 

2. Emphasizing Performance on Key Measures 

State Options for Emphasizing Outcomes 

After the performance measures have been chosen, the next step is to 
decide whether all outcomes are equally important or whether the 
state wants to emphasize key outcomes over other measures. There 
are two ways that this can be done: 

A. Weighting standards differently in calculating incentive 
awards. 

B. Establishing core standards that must be exceeded to 
receive any incentive awards. 



II-8 



Option A: Weighting Standards Differently in Caicuiating 
Incentive Awards 

Intention of Policy. States chose higher weights for standards more 
important to their goals for the JTPA program. In PY 88, some states 
chose foilow-up standards as federal standards but gave these 
standards lower-than- average weights as a transition strategy. 

Past Experience. In PY 88, 25 states explicitly weighted standards 
differently. Generally, states tend to weight the adult standards more 
than the youth standards. 

However, 21 states unintentionally weighted standards differently 
by using a composite measure of the extent to which standards were 
exceeded. As described in detail in Section C, these states used a 
composite measure to aggregate standards that calculated the 
percentage by which each standard was exceeded and summed across 
standards. This procedure inadvertently gave greater weight to 
standards that were easier to exceed by a large amount. In the past, 
this approach gave substantially greater weight to the cost standard. 
In PY 90, states are precluded from basing incentive awards on cost 
standards, so this problem of inadvertent weighting will be reduced 
although not eliminated. Appendix A presents a technical note on 
ways to reduce this unintended weighting of standards. 

Example of Policy. As Colorado moved toward a post- 
program system in PY 88/89, various weights were assigned to 
performance measures in awarding incentive funds. The 
selected standards and weights for each program year are 
outlined below. 
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Weights (%) 


Performance Standard 


PY8S 


PY89 
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1 ^ A 

13.0 
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0,0 
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weuare ionow-up employment rate 


15.0 




Youth positive-termination rate 


40.0 


25,0 


Youth cost per positive termination 


0.0 


0,0 


at follow-up 






Youth employability enhancement 


15.0 





rate (state standard) 



By establishing this weight distribution, Colorado eliminated 
incentive awards for the cost standard and promoted follow-up 
employment as the priority performance measure. 

Option B: Establishing Core Standards 

Intentions of Policy. States have established some standards as 
core standards that must be exceeded before an SDA can receive any 
incentives. States establishing core standards indicate that these 
outcomes are so important that exceeding them is a necessary 
condition for being considered a good performer. As a result, SDAs 
are not allow to tradeoff overperformance on a noncore standard 
against underperformance on a core standard. 

Past Experience. In PY 88, 9 states established core standards. 
The adult and welfare employment standards were frequently 
included in the core, although the youth employability enhancement 
r*ite was included in the core by 4 states. 

Example of policy. Oklahoma required that SDAs exceed six of 
the eight performance standards to be eligible for a "base award." 
The adult entered-employment rate, adult welfare entered- 
employment rate, adult follow-up employment rate, and youth 

22 



9 

ERIC 



11-10 



employability enhancement rate were identified as core standards that 
had to be among the six standards exceeded. SDAs that failed to 
exceed one of these four core standards v ;:e ineligible for the base 
award, which accounted for 50% of the total available incentive 
award. 



Effects of Emphasis on Key Standards 

State efforts to emphasize different outcomes do generally 
increase SDAs' performance on those outcomes. However, the 
increased performance on the emphasized standard sometimes occurs 
at the expense of performance on other measures. This is particularly 
true for the youth measures: emphasis on employment-related 
standards tends to decrease performance on the employability 
enhancement measure and vice versa. It is important that states 
recognize this tradeoff: if a state chooses to weight one standard more 
highly, it may reduce performance on the other types of standards. 

One exception to this tradeoff among measures occurs for the 
earnings standards, including the wage at placement and weekly 
earnings at follow-up. As shown in Exhibit II-2, emphasis on these 
standards increases performance on both earnings measures and 
follow-up employment measures. One hypothesis is that participants 
earning higher wage rates are more likely to stay on the job than those 
earning lower wage rates, resulting in higher follow-up employment 
rates. 

Emphasis on different standards can also affect the types of clients 
enrolled in JTPA and the types of services provided. Placing greater 
weight on the wage standard tends to increase the proportion of adult 
participants receiving classroom training and decrease the proportion 
receiving OJT. However, this shift may partially explain the fact that 
greater emphasis on the wage standard somewhat reduces the number 
of dropouts served because many classroom providers require a high 
school diploma at entry. For youths, greater emphasis on the entered 
employment rate standard tends to reduce service to in-school youths. 

A key finding is the substantial unintended effects of emphasizing 
the cost standards. In PY 86, emphasis on cost standards reduced 
performance on all the outcome measures for adults and service to 
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Exhibit II-2 



SUMMARY OF EFFECTS OF GIVING HIGH WEIGHTS TO 
DIFFERENT TYPES OF STANDARDS 



Adult Earnings Measures (Wage at Placement and Follow-up 

Weekly Earnings) 



■ Adult weekly earnings at 
follow-up 

■ Adult follow-up employment 
rate 

Increases proportion of adults 
receiving classroom training in 
occupational skills and reduces 
proportion receiving OJT 



Intended Effects 



Unintended Effects 



Increases performance on: 
■ Adult wage at placement 



Somewhat reduces service to 
adult dropouts 



Cost Measures 



Intended Effects 



Unintended Effects 



None 



Decreases service to adults and 
youth welfare recipients and 
dropouts 



Increases number of youths 
participating in job search 
assistance 



Decreases average length of 
training for adults 



In PY 86, reduced performance 
on all adult outcome standards 
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welfare recipients, dropouts, and those with other barriers to 
employment. Emphasis on the cost standards also reduced reduced 
provision of longer-term services for adults and increased the 
provision of job search assistance services for youths. For these 
reasons, states are prohibited from basing incentive awards on CO M 
standards, beginning in PY 9Q. 



B. QUALIFYING FOR INCENTIVE AWARDS 

The second major policy element in the state's incentive policy is 
to determine when an SDA's performance is sufficient to qualify for 
incentives. Two provisions in the legislation have affected states' 
policy choices: 

Incentive grants ... shall be distributed among service delivery areas 
exceeding their performance standards in an equitable proportion based on 
the degree by which the service delivery areas exceed their performance 
standards. Section 202(b)(3)(B) 

The Governor shall provide technical assistance to programs which do 
not meet performance criteria. If the failure to meet performance 
standards persists for a second year, the Governor shall impose a 
reorganization plan. Section 106(h)(1) 

Thus, the legislation requires that incentives be given to SDAs 
exceeding performance standards and that sanctions be applied to 
SDAs failing to meet performance standards. However, the 
development of operational definitions of exceeding and failing to 
meet standards has been left to the states. 

Many states have felt a tension between these two requirements. 
On the one hand, they do not want to set the criteria for failing so 
high that SDAs performing at an adequate level fail; on the other 
hand, they do not want to set the criteria for exceeding so low that 
SDAs performing at a marginally adequate level get substantial 
rewards for good performance. Several options to distinguish 
between exceeding and failing to meet standards are described in this 
section. 



11-13 



In establishing criteria about when SDAs qualify for incentives, 
states must develop two separate policies: 

■ How to define exceeding and failing to meet individual standards 

■ How to define exceeding or failing to meet performance standards 
overall, usually based on the number of standards exceeded. 

1. Defining Exceeding and Failing to Meet individual Standards 
Options for Individual Standards 

States have used two basic options in defining whether an SDA's 
performance exceeds or fails to meet an individual standard: 

A. Use a single performance level to separate exceeding the 
standard from failing to meet the standard. 

B. Use a higher performance level to define exceeding the 
standard and a lower level to define failing to meet the 
standard. 
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Option A: Use a Single Performance Level to Separate 
Exceeding from Failing to Meet the Standard 

With a single level separating exceeding the standard from failing 
to meet the standard, an SDA does either one or the other. With this 
"knife edge" approach, there is no middle ground where an SDA 
merely meets the standard (except in the rare event that the actual 
performance is exactly equal to the standard). Consequently, the 
difference in performance between an exceeding SDA and a failing 
SDA might be very small. 

Failing Exceeding 
/ 



Intentions of Policy. States that want all their SDAs to receive 
some 6% funding, often as a buffer against declining 78% allocations, 
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tended to use this "knife edge" approach. "With the level for receiving 
any incentives set at a relatively low level, most SDAs would exceed 
standards and thus qualify for incentive awards. 



Past Experience. A single number was used by 28 states in PY 88 
to define exceeding and failing to meet individual standards. Twenty 
states that set a single level used the model-adjusted performance 
level to separate exceeding from failing to meet standards; the other 8 
states used the lower level of the tolerance range (i.e., model-adjusted 
performance level minus the tolerance range). 

Example of Folicy, Alabama has defined exceeding a standard as 
performance above the lower bound of the tolerance range. SDAs 
that perform below this level are considered to have failed to meet 
the standard. If an SDA fails to meet five or more standards, it is 
ineligible for an incentive award. Those SDAs that meet this 
eligibility criterion receive an award based on the extent to which 
standards are exceeded. 

Option B: Use a Lower Level to Define Failing to Meet 
the Standard and a Higher Level to Define Exceeding the 
Standard 

Defining separate levels for exceeding and failing to meet a 
standard establishes three levels of performance. SDAs with 
performance below the lower level have failed to meet the standard. 
SDAs with performance above the higher level have exceeded the 
standard. In between, SDAs have met the standard and thus are not 
at risk of sanction for that standard, b< : hey have not exceeded the 
standard and thus do not qualify for incentives for their performance 
on that standard. 



Failing 



Meeting 



Exceeding 



/ 



/ 
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Intentions of Policy, Some states used this policy to assure that 
there was substantial difference in performance between SDAs 
exceeding a standard and those failing to meet it Some of these 
states wanted to reserve incentive awards for high performance. For 
example, one state that established a pronounced middle ground 
between failing to meet and exceeding a standard explained its intent 
as encouraging high overall performance without setting up the 
expectation that any single SDA was likely to receive the highest 
possible award on every standard. In this state, every SDA was 
expected to meet every standard, and every SDA typically received 
something in the way of incentive awards for performance on one or 
more standards. 

Past Experience. Fifteen states defined separate ranges for failing 
to meet, meeting, and exceeding each standard in PY 88. Nine of 
these states set a relatively lenient level for exceeding standards by 
using the model-adjusted performance level for the top of the range 
and by subtracting either the tolerance range (7 states) or another 
value (2 states) for the bottom of the range. Two states used the 
model-adjusted performance level plus the tolerance level for top 
range and minus the tolerance level for the bottom range. The 
remaining 5 states took a more stringent approach by using the 
model-adjusted performance level as the bottom of the range. One of 
these states defined the upper end of the range as performance above 
the 50th percentile among the nation's SDAs and defined the lower 
end as performance at or above the 25th percentile (the model- 
adjusted performance level). These values were set using the 
performance ranges presented in DOL's Guide for Setting JTPA Title 
II-A Performance Standards for PY 88. 

Example of Policy. Illinois used the lower and upper limits of the 
tolerance range applied to the model-adjusted standard to distinguish 
failing to meet, meeting, and exceeding a standard. Performance 
below the lover level of the tolerance range constituted failure to 
meet, performance within the tolerance range constituted meeting, 
and performance above the range constituted exceeding the standard. 
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Effects of Alternative Definitions of Exceeding and Failing to 
Meet Individual Standards 

States' choices oi how to define exceeding and failing to meet 
individual standards do not themselves affect SDA behavior 
differently. However, states that choose Option A, a single 
performance level to separate failing to meet from exceeding, are still 
faced with the problem of distinguishing marginally adequate 
performance from exemplary performance. As a result, these states 
are more likely to place substantial emphasis on exceeding standards 
in their formula for calculating the incentive amounts. As we discuss 
in detail in Section C, policies that emphasize exceeding standards 
have the unintended effects of reducing service to the hard to serve 
and reducing provision of basic skills training. 

States that set up ranges for failing, meeting, and exceeding 
standards often are less likely to adopt formulas that continuously 
reward increasingly higher performance with increasingly higher 
incentive payments. Tims, this mechanism can be viewed as an 
alternative way to distinguish marginally adequate from exemplary 
performance that does not lead to strong emphasis on exceeding 
standards. 

2. Defining Exceeding Performance Standards Overall 
Options for Overall Performance 

After determining whether SDAs have exceeded individual 
standards , the next step is to determine whether SDAs' performance 
over all standards is sufficient to be rewarded. Three approaches 
have been used by states: 

A. A minimum number of standards that must be met or 
exceeded. 

B. A minimum number of standards with the provision that 
certain core standards must be met or exceeded. 
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C. A minimum score on an index combining performance over 
the standards. 

Because the federal standards have been reduced from 12 (with 
states selecting 8) in PY 88/89 to a mandatory 6 for PY 90/91, states 
will need to revise their incentive policies, particularly the minimum 
number of standards or the core standards required to receive 
incentive awards. In developing new policies, states should be careful 
to meet the federal requirement that incentives be given to SDAs 
exceeding all of the federal standards. Thus, the number of standards 
that must be met to qualify for incentives cannot exceed the number 
of federal standards (6 in PY 90/91), and state standards cannot be 
used as core standards that must be met to receive incentives. 

Option A: Setting a Minimum Number of Standards That 
Must Be Met or Exceeded 

Intentions of Policy. States require meeting a high number of 
standards to promote overall program success and limit distribution of 
incentive awards to an "elite" set of SDAs with exemplary 
performance. States require meeting fewer standards to allow SDAs 
flexibility to trade off performance among the standards, thereby 
allowing SDAs to focus programs to meet local needs and goals. 

Past Experience. In PY 88, 13 states had no gate based on 
performance; in these states, SDAS exceeding j ns t QSfi Ef andard could 
gSl fl P ip^ntive award. Thirty-one states required that a minimum of 
4 to 8 standards be met. The typical state required that five (i.e., 
more than half) standards be met; however, 7 states required that all 
^ Mf ^ nhtfl nH fl rd s bemet before an SDA could qualify for any 

incentives. 

Example of Policy. In Idaho, SDAs must have exceeded five of the 
eight federal standards to be eligible for incentive awards. The state 
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did not identify specific standards that must be exceeded; any five 
standards were acceptable. 

Option B: Requiring That Core Standards Be Exceeded 

Intentions of Policy. States establish core standards to emphasize 
specific outcomes, as discussed in Section A above. The greater the 
number of standards in the core, the less an SDA's flexibility to trade 
off performance among the standards. 

Past Experience. In PY 88, 9 states established three to five core 
standards that must be met or exceeded. These states often required 
meeting or exceeding an additional number of unspecified standards 
as well. The typical policy was meeting or exceeding at least five 
standards, including three core standards. 

Example of Policy. Virginia's incentive award policy required 
SDAs to exceed five of the eight selected federal performance 
standards to be eligible for an incentive award. Further, three of the 
five standards had to be the adult entered-employment rate, the adult 
welfare entered-employment rate, and the adult cost per entered 
employment. SDAs that failed to exceed the three core standards 
were ineligible for an incentive award. 

Option C: Minimum Score on a Composite Index of 
Performance 

Several states have created a composite index of performance that 
sums the extent to which an SDA has exceeded its performance 
standard (typically the percentage by which each standard is 
exceeded) across all measures. (Composite indexes are discussed in 
detail below.) These states then set a minimum level for this index 
that SDAs have to meet to qualify for incentives. 
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Intentions of Policy. States use a minimum level on a composite 
index to allow SDAs flexibility. As long as the SDA's overall score 
meets the minimum level, it is allowed to trade off exemplary 
performance on one standard with merely adequate performance on 
another. 

Past experience. Twenty-four states used a composite index of 
performance in PY 88. Most of these states required that the overall 
score be positive, so that, across all standards, the amount by which an 
SDA exceeded performance standards was greater than the amount 
by which it failed to meet standards. 

Example of Policy. Missouri developed a composite index of 
performance to distribute incentive awards. For each of the eight 
federal standards, the state calculated the percentage by which the 
SDA exceeded or failed to meet the standard. For each SDA, the 
overall average was then calculated by summing the difference 
between performance and the standard for each federal measure and 
dividing by 8. SDAs with an overall average of zero or less were 
ineligible for incentive awards. 

Effects of Alternative Criteria for Exceeding Standards 
Overall 

The difference between these three options is the extent to which 
the options emphasize specific standards. As discussed above, 
establishing core standards does increase SDAs' performance on 
those outcomes. The intended and unintended effects of emphasizing 
different outcomes were presented in Section A. 

Setting a minimum number of standards that must be exceeded is 
neutral with respect to the different standards and thus gives SDAs 
flexibility in how they meet this criterion. 

] "> 
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Although nominally neutral with respect to different standards, 
establishing a composite index of performance has inadvertently given 
more emphasis to standards that are easier to exceed by a wide 
margin. In the past, this approach has given greater weight to the cost 
standard, resulting in substantial unintended effects. Among the six 
federal standards for PY 90, this index gives greater weight to youth 
measures, particularly the youth employability enhancement rate, 
because this measure is easier to exceed by a wide margin than, for 
example, the earnings at follow-up measure. 

Regardless of which option is chosen, the number of standards 
required (or the level of the composite index that must be achieved) 
has the major influence on SDA behavior. Requiring that a large 
number of standards be exceeded is a policy that places a strong 
emphasis on performance. Section C presents the effects of such 
policies, which tend to discourage service to the hard to serve and 
reduce provision of basic skills training. 

Further, requiring that a large number of standards be exceeded to 
qualify for incentives has the intended effect of increasing 
performance on termination-based measures, although the effect on 
follow-up performance is much weaker. Further, requiring that a 
large number of standards be exceeded has a particularly strong effect 
in discouraging service to welfare recipients. States that want to 
encourage SDAs to coordinate services with the JOBS program or to 
serve welfare recipients in general, therefore, should avoid a policy 
requiring that a large number of standards be exceeded before SDAs 
qualify for incentives. 



C. CALCULATING INCENTIVE AWARDS 

The third major element of inceiuive policies is the procedure 
used to calculate incentives for those SDAs that qualify for awards. 
Simplicity is the key. Incentive systems are most effective in 
motivating SDA behavior when they can be easily understood by SDA 
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staff. Complicated policies are subject to unintended effects, both 
because they do not clearly convey state goals to SDAs and because 
they may contain hidden incentives not intended by the state. 
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States generally made tradeoffs among three goals in choosing 
their policies for calculating incentive payments. First, some states 
wanted to distinguish marginally adequate performance from superior 
performance in awarding incentives. Second, some states viewed 
incentive payments as funds to help buffer against declining 78% 
funding and so generally wanted to maximize the amount of incentive 
funds that SDAs received. Third, some states wanted SDAs to be able 
to predict how much incentive funding they would receive so that the 
SDAs could better plan how to use those funds. These considerations 
led to a wide diversity in procedures to calculate incentive funds. 

States must make choices about several elements in their incentive 
calculation policies: 

■ Calculation approach. 

■ Amount that SDAs receive for marginally exceeding their 
standards. 

■ Procedures for rewarding performance beyond the standard. 

■ Whether there is a cap on rewarded performance beyond which no 
additional incentives are received. 

■ Whether SDAs compete for the size of the incentive award. 

■ Whether adjustments are made for the size of the SDA. 

The options for each of these policies are presented below. It is 
difficult to separate the independent effect of each of these options on 
participants served, services offered, and performance achieved. In 
general, however, options that overemphasize exceeding standards 
have substantial unintended effects. The end of this section 
summarizes the effects of such policies and presents examples of 
combinations of policies that have particularly strong unintended 
effects. 
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Policies for Calculating Incentives 
1. Calculation Approach 

Options for the Calculation Approach 

States used two basic approaches to calculate incentive awards: 

A. Separate reward pool for each standard. 

B. Composite measure of performance across all standards. 

Most states use one of these procedures, although a few use both 
separate pools and a composite measure for different portions of their 
incentive funds. These basic approaches are discussed below. 

Option A: Separate Reward Pool for Each Standard 

Many states have created a separate pool of incentive funds for 
each federal and state standard. By rewarding performance on each 
standard separately, the incentive formula becomes relatively easy for 
both state and SDA staff to understand. 

Having a separate pool of incentive funds for each standard also 
makes explicit the weight that is placed on each standard. Thus, 
standards can be weighted by setting aside an appropriate percentage 
of incentive funding for each standard. 

Although rewards for performance on each standard can be 
calculated through any of the methods discussed below, the 
development of a separate pool of incentive funds for performance on 
each standard is easier and less trouble prone than developing 
composite formulas. 

Intentions of Policy. Some states have chosen separate pools of 
incentive funds to highlight the different weights placed on different 
standards. States that view incentive funds as supplementary funding 
for all SDAs are also more likely tu use separate pools so that SDAs 
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will be able to predict more easily how much in incentive funds they 
will receive. 



Past Experience. In PY 88, 18 states established a separate pool of 
incentive funds for each standard. 

Example of Policy. New Hampshire developed separate reward 
pools for each standard based on the following weights: 

Adult entered-employment rate 

Adult average wage at placement 8 

Adult welfare entered-employment rate 10 

Youth entered-employment rate 15 

Youth employability enhancement rate 1 

Adult follow-up employment rate 18 

Adult follow-up weekly earnings 20 

Adult welfare follow-up employment rate 1 8 

The available incentive award funds were divided among the 
performance standards according to the weight distribution above. 
Hence, New Hampshire encouraged SDAs to secure and retain 
quality placements for participants by placing heavy emphasis on the 
follow-up standards. 

Option B: Composite Measure of Performance 

An alternative is to award incentives based on a composite 
measure of performance across all the standards. Composite 
measures tend to be more complicated to develop, and the implied 
incentives can be difficult to understand. Consequently, a number of 
states have developed formulas that do not truly reflect the intended 
state goals. 

Many states have made an explicit decision to weight standards 
equally and used the unweighted sum of the percentages by which 
each standard is exceeded as the composite performance measure. 
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Although intuitively appealing, this procedure has the unintended 
effect of emphasizing standards that are easy to meet by a large 
percentage. The cost standards were much easier to exceed by a large 
percentage in PY 88/89 than any of the other standards except the 
youth employability enhancement rate. Thus, this policy 
unintentionally led to an overemphasis on cost standards. Because 
cost standards can no longer be used to determine the size of 
incentive awards, the problem raised by this formula is considerably 
reduced. However, some differences among standards remain. 

To show these differences, the percentage above standard was 
calculated for an SDA performing at the 75th percentile on every 
standard: 

% Above Standard 

14 
15 
24 
19 
65 
36 

These data show that it is much easier to exceed the youth 
measures (especially the employability enhancement rate) by a large 
percentage than any of the other measures, while it is somewhat 
easier to exceed the welfare measures by a large percentage than the 
adult measures. States using a composite measure that does not 
explicitly weight standards differently should recognize, therefore, that 
they are implicitly placing high weight on the youth outcomes and low 
weight on the adult outcomes. Appendix A presents a method to 
correct for this unintentional weighting. 

Intentions of Policy. Some states that view incentive funds 
primarily as rewards for exemplary performance have chosen a 
composite-index approach. Others use this approach because they 
intended to treat each standard equally and allow SDAs to trade off 
performance among the different standards. 



Adult follow-up employment rate 
Adult follow-up weekly earnings 
Welfare follow-up employment rate 
Welfare follow-up weekly earnings 
Youth employability enhancement rate 
Youth entered-employment rate 
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Past Experience. Twenty-four states used a composite measure of 
performance to calculate incentive funds in PY 88. 

Example of Policy. Rhode Island established a composite measure 
to evaluate SDA performance as follows: 

■ The state calculates the percentage difference between actual 
performance and the standard for each of the eight performance 
measures for each SDA. 

■ The positive and negative percentage differences for each of the 
eight standards are summed to an SDA "net score." 

■ The net scores for the three SDAs are totaled and equated to 
100%. 

■ Each SDA's positive net score is expressed as a percentage of the 
total score, thus representing the SDA's competitive share of 
available incentive award funds. 



Effects of Alternative Calculation Approaches 

The choice of establishing a separate pool for each measure or 
of using a composite measure does not in itself affect SDA 
behavior. However, separate pools make the weights placed on 
standards more explicit and visible to SDAs and, therefore, are 
likely to communicate more clearly the state's priorities. 

2. Amount Received for Marginally Exceeding Standards 

States next must decide how the amount of incentive awards 
will be tied to the degree to which performance standards are 
exceeded. This is a very important policy decision because a 
strong emphasis on exceeding standards can result in substantial 
unintended effects on the types of participants enrolled and on 
the intensity of services provided in JTPA. 

An important consideration is whether SDAs will receive any 
incentive awards for marginally exceeding their standards, that is, 
for performing at a level slightly above the level defined as 
exceeding standards. 
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The current JTPA legislation states that incentives must be 
based on the degree to which the standards are exceeded. If 
states wish to reduce the emphasis on exceeding standards by 
awarding incentives for just marginally exceeding the standards, 
they can meet this legislative requirement by awarding a portion 
of incentive funds for marginally exceeding standards ("base" 
awards) and an additional portion for higher levels of 
performance ("bonus" awards). 

Intentions of Policy. The portion of incentive funds awarded for 
marginally exceeding standards reflects the state's emphasis on 
meeting standards relative to exceeding standards. Some states have 
chosen to award at least a portion of incentives for marginally 
exceeding standards to reduce the risk to SDAs of serving hard-to- 
serve groups. If SDAs only just exceed standards, they will still 
receive incentive awards. Other states have chosen this policy to 
make sure that SDAs receive maximum incentive funds to supplement 
their 78% funds. 

Past Experience. In PY 88, 30 states awarded a portion of incentive 
funds for marginally exceeding standards. Among these states, the 
amount awarded for marginally exceeding ranged from 5% to 100% 
of incentive funds, with the typical state awarding 50%. 

Example of Policy. Texas rewarded SDAs that marginally 
exceeded their standards. In this system, 50% of available incentive 
awards were distributed to SDAs with performance marginally (less 
than 10%) above the standard. Thirty-five percent of incentive funds 
were distributed to SDAs with performance between 10% and 14.9% 
above the standard, and the remaining 15% were awarded to SDAs 
with performance 15.0% or more above the standard. Although this 
incentive policy rewards SDAs with higher levels of performance, half 
of all available incentive funds are distributed to SDAs with marginal 
performance as a base award. 

Effects of Policy. Awarding a substantial amount of incentive 
payments based on marginally exceeding performance standards 
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reduces the emphasis on exceeding standards. As discussed below, 
policies that reduce the emphasis on exceeding performance 
standards increase service to the hard to serve and increase the 
provision of more intensive services. 

3. Rewarding Performance Beyond the Standard 
Options for Rewarding Performance 

Because of the current legislative requirement that incentives be 
awarded based on the degree by which the SDAs exceed their 
performance standards, states have developed procedures to reward 
performance beyond the standard. States have chosen two 
approaches to awarding incentives for performance beyond the 
standard: 

A. Tiered systems 

B. Continuous systems. 

Each of these approaches is discussed below. 

Option A: Tiered Systems 

Some states use a step function or tiered system for rewarding 
performance beyond the standard. Tiered systems can be used with 
either a separate award pool for each standard or with a composite 
performance index. (Tiered systems also naturally lend themselves to 
rewarding performance for marginally exceeding the standard and to 
capping rewarded performance, although neither is a necessary 
consequence of tiered systems.) 

Performance in the lowest tiei (possibly at a level marginally 
exceeding the standards) results in the SDA's earning a certain 
proportion of its total potential reward. Performance at one or more 
intermediate levels increases the SDA's share of its total potential 
reward. Performance at or beyond a final performance level (the 
"cap") results in the SDA's having maximized its incentive award for 
that standard. 
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The tiered system is often used in combination with a separate 
pool of incentive funds for each standard but it can also be used with a 
composite measure of performance as well. For example, one way to 
use a tiered system with a composite index is to award "points" for 
each standard based on the tier achieved and sum those points over 
all standards. Incentive awards would then be based on total points. 

The boundaries between the tiers can be set using several 
approaches: 

■ The performance ranges given in the DOL technical assistance 
guide. For example, the lowest tier could range from the standard 
to the 50th percentile of perfoimance, the second aom the 50th to 
75th percentiles, and the upper tier could be the 75th percentile 
and above. 

■ A given percentage above the standard. For example, one state 
that used a three-tiered award system in PY 86 awarded 50% of 
the possible incentive award on each standard to SDAs with 
performance between 100% and 110% of their standard. SDAs 
with performance between 1 10% and 120% of their standard 
received 80% of their possible incentive award. The incentive 
award formula was capped at 120%, so that SDAs that performed 
at Oi above 120% of their standard earned 100% of their possible 
incentive award. 

Intentions of Policy. States chose a tiered approach for several 
reasons. First, this approach made it easier for SDAs to predict 
how much in incentive funds they would receive because they 
could estimate in which range their performance would be likely 
to fall. Second, states that wanted to make it easier for SDAs to 
earn their maximum incentives used the tiered approach and jet 
relatively low levels of performance for the upper tier. 

Past Experience. In PY 88, 19 states used a tiered approach to 
reward performance beyond the standards. 

Example of Policy. Minnesota uses a tiered performance 
award system with tolerance factors to establish the tiered 
boundaries. Separate pools of equal size are created for the eight 
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federal measures from the available incentive funds. Incentive 
awards are based on the SDA's size and the extent to which the 
SDA exceeds the standard. A three-tiered distribution approach 
has been established in which increments of 60%, 30%, and 10% 
of available incentive funds are awarded. SDAs that perform 
within the first tier receive 60% of the award; those that perform 
within the second receive 90% of the award; and those that 
perform in the third tier receive the full award. 

Option B: Continuous Systems 

The second approach to rewarding performance beyond the 
standard is to use a continuous measure of performance. 
Common measures include: 

■ The percentage by which the standard is exceeded 

■ The amount by which the standard is exceeded 

■ The percentage relative to the best-performing SDA. 



Intentions of Policy. Some states have intentionally chosen a 
continuous award system to maximize incentive awards to exemplary 
performers. Other states have chosen this approach as a technical 
solution to the legislative requirement * l t ; ncentives be based on the 
degree to which standards are exceedeo fa these states, the emphasis 
on exceeding standards is unintended. 

Past Experience. Twenty-three states chose a continuous reward 
system in PY 88. 



Example of Policy. North Carolina has established a continuous 
incentive award system for SDAs that exceed five of the eight 
performance measures. A • ^rformance index is created for each SDA 
by calculating the percentage by which each standard was exceeded, 
summing over the eight standards and dividing by 8. The sum of the 
indexes is then totaled. The SDA share of available incentive awards 
is determined by dividing the SDA index by the sum of the indexes 
across all SDAs and multiplying this fraction by the available award 
pool. 
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Effects of Alternative Calculation Procedures 

Tiered systems are easy to understand and lend themselves to 
designing incentive award systems that do not overemphasize 
exceeding standards. The extent of emphasis on performance 
depends on where the tier boundaries are set and how the award 
increases as performance steps from one tier to the next. The 
disadvantage of the tiered approach is that two SDAs can receive very 
different awards with similar performance if they perform just above 
and just below a tier boundary. 

The problem with using a continuous system is that by itself it 
provides no incentive payments to SDAs just marginally exceeding a 
standard, and it places substantial emphasis on exceeding standards by 
providing increasingly higher incentive awards for increasingly higher 
performance. 

Both rewards for marginally exceeding standards and capping of 
rewarded performance can be added to continuous formulas. For 
example, SDAs could be awarded 10 points for just exceeding the 
standards plus one point for every percent the standard was exceeded. 
Alternatively, a separate pool of incentive funds can be used to 
reward SDAs marginally exceeding standards. Capping can be 
accomplished by, for example, giving no additional incentives for 
performance above the 80th percentile. 

4. Caps on Rewarded Performance 

Another important element in calculating incentives is whether 
there is some maximum amount of performance above the standard 
that will be rewarded. Performance above that amount would not 
result in any additional incentives. 

Intentions of Policy. Some states place a cap on rewarded 
performance to reduce the emphasis on overperformance. 
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Past Experience. In PY 88, 21 states capped rewarded 
performance, with the average cap about 16% above the standard. 

Examples of Policy. The distribution of incentive award grants in 
Oregon is an example of a tiered system with a cap. To illustrate 
Oregon's system, the performance needed to receive incentive awards 
for the adult entered-employment rate (AEER) and welfare entered- 
employment rate (WEER) are presented below: 

Percent of Full Incentive Award 

80% 85% 90% 95% 100% 
Standard Performance 

AEER 68% > 68 to 71 > 71 to 73 > 73 to 76 > 76 to 79 > 79 

WEER 56% > 56 to 61 > 61 to 66 > 66 to 71 > 71 to 76 > 76 

SDAs received their full incentive award at the 100% performance 
level indicated above. Excessive overperformance was not rewarded 
through this policy. 

Colorado designed a continuous incentive reward system with a 
cap. This policy set aside a portion of available incentive funds for 
each standard based on a weighted distribution process. Colorado 
defined the range of rewardable performance for PY 88 and 89 as a 
percentage of the model-adjusted standard; it extended from the 
standard to a level 10% above the standard. Performance at a level 
above the cap did not result in additional incentive funds. Incentive 
awards were based on actual performance within the range. For 
example, if an SDA's standard for weeks worked at follow-up was 8 
weeks, the range of rewardable performance for that SDA extended 
from 8 to 8.8 weeks. If the SDA just met its standard (8 weeks), it was 
ineligible for any incentive awards for that standard. If the SDA's 
performance was 8.4 weeks, the SDA was eligible for half of its award 
pool. Performance at a level of 8.8 weeks or higher resulted in the 
SDA's receiving its full award. 
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Effects of Policy. Capping performance reduces the emphasis on 
exceeding standards by eliminating the incentive to perform beyond 
the cap. 



5. Competition Among SDAs 

Formulas for calculating incentive awards can be structured to 
incorporate varying degrees of competition among SDAs. In a 
noncompetitive policy, an SDA's award is not affected by the 
performance of other SDAs in the state. Thus, the SDA can calculate 
its award without knowing the performance of other SDAs. In a 
noncompetitive system, the portion of an SDA's maximum incentive 
awards not earned by the SDA is used for purposes other than 
incentive awards (e.g., technical assistance, service to hard-to-serve 
groups). 

In a moderately competitive system, awards not earned by SDAs 
are distributed to SDAs based on performance. In this case, the SDA 
can determine its base award without knowing the performance of 
other SDAs, but not its secondary award from the unearned pool. 

In a highly competitive system, even the base award depends on 
the performance of other SDAs. Policies that distribute funds based 
on the SDA's share of total scores on a composite index and policies 
that compute performance relative to the best SDA are highly 
competitive. 

Intentions of Policy. Some states have intentionally emphasized 
competition among SDAs so that the awards represent exemplary 
performance among SDAs in the state. Other SDAs have chosen a 
noncompetitive process so that SDAs can better predict how much in 
incentive funds they will receive and thus can better plan for the use 
of those funds. 
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Past Experience. In PY 88, 18 states had noncompetitive incentive 
award systems, while 27 states had either highly competitive or 
moderately competitive systems. 

Examples of Policies. Wisconsin's incentive award system uses a 
noncompetitive tiered approach to determine the amount of an SDA's 
incentive award, with a moderately competitive approach used to 
distribute unearned funds to SDAs with outstanding performance. To 
be eligible for an award, an SDA must first exceed three adult and 
two youth standards. Fifty percent of the full SDA award will be 
distributed to SDAs that exceed their standards by less than 5%. 
SDAs that exceed their standards by at least 5% but less than 10% 
receive 75% of their full awards, while SDAs that exceed their 
standards by 10% or more are eligible for the full award. 

Funds not distributed by this noncompetitive base system are 
allocated equally among SDAs that exceeded performance by 10% or 
more. If none of the SDAs exceed their standards by more than 10%, 
the remaining funds are distributed equally among those SDAs that 
exceed standards by 5%; and if none exceed 5% of standards, the 
remaining funds will be distributed to those SDAs exceeding their 
standards. 

Effects of Policy. Competitive incentive formulas increase the 
emphasis on exceeding standards. 

6. Adjustments for Size of SDA 

Intentions of Policy. States that adjusted for SDA size usually 
wanted to make the incentive awards of equal importance to SDAs of 
varying size in the state, by making it possible for them to supplement 
their 78% formula funds by equal percentages with equivalent 
performance. States that did not adjust the incentive awards for SDA 
size usually wanted to make the absolute size of the potential 6% 
award for small SDAs large enough to be attractive to them. Several 
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of these states capped the size of the maximum possible award an 
SDA could earn (e.g., at 50% of 78% formula funding), so that a small 
SDA would not receive a 6% award totally out of proportion to its 
78% allocation. 

Past Experience. In PY 88, 29 states based all of their 6% incentive 
awards on SDA size; 9 states states made no adjustment for the size of 
the SDA; while 6 used a mixed approach. 

Example of Policy. One of the two primary factors Washington 
State used to allocate incentive awards was the size of the SDA. State 
policymakers provided the following justification for this approach in 
the GCSSP: 

Size has been included in the formula because it takes more money for 
a large SDA to improve its performance by a given amount than it does for 
a small SDA. Large SDAs would have to place more participants to 
increase their entered employment rate by one percent than would a small 
SDA. 

Similar explanations were provided by other states that chose to 
allocate available incentive funds based on the size of the SDA's Title 
II-A allocation. 

Effects of Policy. Adjustments for SDA size make 
performance equally important for all SDAs, although very small 
SDAs may find potential awards too small to use effectively. 
Using no adjustment for SDA size can make performance 
incentives essentially irrelevant for the largest SDAs in the state. 

Effects of Emphasizing Exceeding Standards 

Several options available to distribute incentive awards place 
an emphasis on exceeding performance standards. States often 
have chosen several of these options and as a result may have 
unintentionally placed an undue emphasis on exceeding standards 
at the expense of other program goals. 
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Policies that place a strong emphasis on high performance 
include: 



■ Requiring that a large number of standards be exceeded to qualify 
for incentive awards. 

■ Not awarding any incentives for marginally exceeding standards. 

m Rewarding higher performance with greater incentives without any 
cap on rewarded performance. 

■ Using a competitive system to reward performance so that an 
SDA's incentive award depends on the performance of other 
SDAs in the state. 

The effects of placing a strong emphasis on exceeding 
standards are summarized in Exhibit II-3. Emphasizing exceeding 
standards does increase performance on employment-related 
outcomes. This is particularly true for standards for entering 
employment at termination. The effects on follow-up 
employment measures are smaller. Emphasizing exceeding 
standards has no significant effect on the earnings measures. 
Thus, there is evidence that these policies increase the quantity 
but not the quality of placements . 

Emphasis on exceeding standards also has substantial unintended 
effects on the types of individuals participating in JTPA and the 
services provided. Strong emphasis on exceeding performance 
standards reduces services to welfare recipients and minorities for 
both adults and youths and reduces services to older adults and to in- 
school youths. Further, these policies increase the employment focus 
of training, reducing the number of adults and youths who receive 
basic skills remediation through JTPA. 

A hypothetical example of an incentive policy that would place the 
maximum emphasis on exceeding standards would be one in which: 

■ SDAs must exceed all performance standards to qualify for 
incentives. 
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Exhibit II-3 



SUMMARY OF EFFECTS OF EMPHASIS 
ON EXCEEDING STANDARDS 



Intended Effects 

Increases performance on 
termination-based employment 
measures 

Somewhat increases 
performance on follow-up 
employment measures 



Unintended Effects 

Reduces service to adults who 
are welfare recipients, 
minorities, and older workers 

Reduces service to youths who 
are welfare recipients, 
minorities, or in school 

Reduces the proportion of 
participants receiving basic skills 
training for adults and youths 
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■ A composite measure is used that computes the percentage by 
which the SDA exceeds each standard and sums across all 
standards. The amount of incentive received is based on the score 
on this index. SDAs that marginally exceed a standard, therefore, 
receive little if any incentive funds for that standard. 

■ There is no cap on rewarded performance. SDAs that score 
increasingly higher on this index receive increasingly more 
incentive funds. 

■ The amount of incentive funds each SDA receives is determined 
competitively. The proportion of incentives an SDA receives is 
determined by dividing its index score by the sum of all scores for 
SDAs in the state. Thus, even if an SDA exceeded all standards by 
a wide margin, if other SDAs perform at an even higher level, that 
SDA's incentive funds are reduced. 

The above example represents an extreme case, not an actual state 
policy. However, states with incentive policies closely resembling this 
example should carefully consider the unintended effects that could 
result by placing such a strong emphasis on exceeding performance 
standards. 

D. CONDITIONS PLACED ON INCENTIVE FUNDS 

The final major element in the development of incemive policy is 
to determine whether any special conditions should be placed on 
incentive funds, compared with regular 78% funds. Two related 
policies have been established by states: 

■ Exempting incentive funds from performance standards 

■ Requiring incentive funds be used for specific purposes. 

1. Exempting incentive Funds from Performance Standards 

Beginning in PY 88, DOL established the policy that states may 
exempt incentive funds in the application of performance standards. 
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Thus, states may adopt the policy that individuals served with 
incentive funds will not be included in calculating the SDA's 
performance. The characteristics and outcomes of those individuals, 
however, must be reported on the Job Training Annual Status Report 
so that DOL will have an accurate account of the types of individuals 
served with Title II-A funds. States that chose this policy either 
applied the exemption of incentive funds statewide or allowed SDAs 
the option of exempting these funds from performance standards. 

Intention of Policy. States have chosen to exempt incentive funds 
for two reasons. First, they want to encourage SDAs to take the risk 
of establishing innovative programs or serving hard-to-serve groups 
and believe that exempting some funds from performance standards 
will allow SDAs to set up such programs and see if they can be 
successful. Second, some states feel that incentive funds will be more 
valuable to SDAs if SDAs have maximum flexibility in how the funds 
are used. 

Past Experience. In PY 88, 20 states adopted a policy to exempt 
incentive funds from performance standards: 10 states excluded 
incentive funds for all SDAs in the state, and 10 gave SDAs the option 
of excluding incentive funds from performance standards. 

Examples of Policy. Tennessee had a statewide policy exempting 
programs funded with incentive funds from performance standards. 
This policy was instituted to encourage SDAs to serve higher-risk 
groups. Examples of programs supported with exempted 6% funds 
include training for teenage parents and basic skills training for 
dropouts. 

Utah gave SDAs the option to include or exclude 6% funds from 
performance-standard calculations. Three of the nine Utah SDAs 
opted to exclude the incentive awards from performance for PY 
88/89. Restrictions were not placed on the use of exempted 6% 
funds. 
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Effects of Policy. SDAs have reacted to the policy of exempting 
incentive funds from performance standards in a variety of ways. 
Many SDAs have used exempted funds to establish programs for 
riskier participants or to fund innovative programs that need time to 
work out problems. Examples include an alternative high school for 
at-risk youth and an innovative program to help participants establish 
their own small businesses. Other SDAs have made use of the 
flexibility of the funds to purchase computerized equipment for 
learning centers. 

Some SDAs have not reacted differently to the fact that incentive 
funds are exempt from performance standards. They use incentive 
funds to supplement 78% funding declines to keep programs funded 
that otherwise would have been cut. A few SDAs, however, have used 
exempted incentive funds to support politically influential providers 
that were not otherwise achieving performance standards. 

Overall, the impact of exempting incentive funds from 
performance standards is to increase the number of hard-to-serve 
individuals served, particularly welfare recipients. 



2. Requiring Incentive Funds to Be Used for Specific Purposes 

Intentions of Policy. States that adopted the policy to exempt 
incentive funds added the requirement that the funds be used for 
specific purposes to guard against unintended effects of the exemption 
policy, such as the funding of underperforming providers. 

Past Experience. In PY 88, 20 states imposed restrictions on the 
use of incentive funds. Virtually all these states required incentive 
funds to be used for serving hard-to-serve groups, and 8 states also 
required that these funds be used for specific types of training, most 
commonly basic skills remediation. 
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Examples of Policy. Nevada has a statewide policy that exempts 
incentive funds from performance standards but requires the awards 
to be used for hard-to-serve groups with multiple barriers to 
employment. Examples of programs funded with exempted incentive 
funds include the following: 

■ Teen Parent Program-Participants were required to be single 
heads of household, 18 to 21 years of age, with at least one 
dependent child. 

■ Staff Specialist-Funds supported outreach and recruitment in 
high- unemployment areas for statewide youth alternative 
education programs. 

■ Remedial Education Programs-Supported tutors for youths at risk 
of dropping out in rural school districts. 

■ Juvenile Justice-Provided supportive assistance and training 
alternatives to youths in the juvenile court system. 

New Hampshire's incentive policy does not exclude incentive 
funds from performance standards but does require incentive awards 
to be used to serve the high-risk JTPA population. 

Effects of Policy. Adding the requirement of how incentive funds 
should be used did tend to reduce the unintended effects that might 
result from exempting those funds from performance standards. 
Thus, SDAs in states that required funds to be used for services to the 
hard to serve were more likely to fund innovative programs and less 
likely to fund underperforming but influential providers. 
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Ill EXAMPLES OF INCENTIVE POLICIES THAT SUPPORT 

STATE GOALS 



A. INTRODUCTION 

The premise of this guide is that incentive policies can be a 
powerful tool for states to further their JTPA program goals. The 
previous section presented the options used by states in developing 
aspects of their incentive policies and the effects of those options. 
This section presents examples of how some states have chosen 
among these options to develop incentive policies to further JTPA 
program goals and priorities. 

These examples show how a coherent set of program objectives 
underlie the development of incentive policies in these states, and 
how the design elements described in Section II can work together. 
These policies are not presented, however, as models appropriate for 
all states. The purpose of this section is not to encourage states to 
develop identical policies but to help states develop incentive policies 
appropriate to their own policy objectives. 

The examples presented are varied in terms of their policy goals 
and incentive strategies. Each of the states, however, has attempted 
to achieve a balance between rewarding good performance and 
encouraging SDAs to address the employment barriers of hard-to- 
serve groups. 

B. GEORGIA: REWARD COMMENDABLE PERFORMANCE 
WITHOUT UNINTENDED EFFECTS 

Policy Objectives 

Georgia's performance standards and incentive policies have been 
designed to reward commendable performance, rather than minimally 
acceptable performance. However, the staie has carefully designed its 
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incentive award policy to prevent incentives from driving the JTPA 
system toward higher performance at the expense of other program 
goals. 

The key elements of the state's strategy for furthering these goals 
include: 

■ Establishing a large range of performance between failing to meet 
the standards and exceeding the standards, with the expectation 
that, while all SDAs will meet the standards, not every SDA will 
exceed most or all standards. 

■ Creating a tiered system for allocating rewards that caps the award 
for exceeding each standard at a specific performance level, limits 
competition among SDAs, and ensures that each standard receives 
equal weight in practice as well as in theory. 

■ Assigning a zero weight to the cost standards in the incentive 
allocation formula to prevent an emphasis on quantity of services 
rather than quality of services. 

Emphasis on Different Outcomes 

During PY 88, Georgia selected 10 of the 12 possible federal 
standards. Eight of these standards were implemented as mandatory 
federal standards (the basis for potential sanctions), and two 
standards that had not previously been implemented in Georgia were 
implemen.ed as state standards. Starting in PY 88, Georgia gave the 
cost standards zero weight. The remaining eight standards were given 
equal weight in allocating incentive awards among SDAs. 

Other than designating two additional federal standards, no state 
perforr mce standards were implemented by Georgia for PY 88, 
although expenditure rates were incorporated into the incentive 
formula. 
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Qualifying for Incentive Awards 



Three ranges of performance were defined: failing to meet, 
meeting, and exceeding each standard. SDAs with performance 
below the model-adjusted standard (approximately the 25th percentile 
of the national performance) were considered to have failed to meet 
the standard. Those with performance above the model-adjusted 
standard but below an upper level were considered to have met the 
standard. This upper level was set to be equivalent to performing 
above the 50th percentile of national performance. Only SDAs 
performing above the 50th percentile were considered to have 
exceeded the standard. 

To qualify for incentive awards, an SDA must meet at least six of 
the remaining eight standards and must ex££ei at least one standard. 



Calculation of Incentive Awards 

The state used a tiered approach to reward performance on each 
standard. SDAs earn one point for exceeding each standard (above 
the 50th percentile) and a bonus point for performing at a level 
greater than the 75th percentile. Thus, rewarded performance is 
capped. By using a tiered system based on percentiles of national 
performance to develop a composite measure and by capping 
rewarded performance, Georgia avoids placing unintended emphasis 
on those standards that are easier to exceed by large amounts. 

The incentive awards are allocated among SDAs relative to the 
total number of performance points they have earned across all 
standards except the cost standard. The size of the incentive award 
received by each SDA is adjusted for the size of the SDA (measured 
by 78% allocation) and the rate of JTPA expenditires. 
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Conditions Placed on incentive Funds 

The state places no restrictions on how SDAs can spend their 6% 
incentive awards. The state has not chosen to exempt 6% incentive 
funds from the performance standards. 

C NEW YORK: FOCUS EFFORTS ON LONG-TERM WELFARE 
RECIPIENTS AND ENCOURAGE INTENSIVE SERVICES 

Poiicy Objectives 

In PY 89, New York reshaped its incentive awards policy to 
further a state goal of serving long-term welfare recipients. These 
changes in the incentive policy shifted the state's strategy for 
providing leadership to SDAs in two ways: 

■ It focused attention on the single hard-to-serve group of highest 
priority rather than establishing many different hard-to-serve 
target groups. 

■ It emphasized that the bulk of the 78% formula Title II-A 
allocation, not just incentive funds, should be used for programs 
for the hard to serve. 

Among the key elements of the state's current design strategy are 
the following features: 

■ Establishment of a new incentive policy that distributes 50% of 
available incentive funds to SDAs for demonstrated service and 
performance to long-term AFDC recipients (and other hard-to- 
serve adults and youths based on SDA proposals) in 78%-funded 
programs. 

■ Deemphasis of cost standards through zero weighting in the 
incentive award formula, to encourage the provision of intensive 
services. 

■ Design of a noncompetitive allocation formula that caps incentive 
awards at 110% of each standard to deemphasize performance in 
excess of the standards. 
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■ SDA option to exempt incentive funds from performance 

standards to encourage SDAs to undertake innovative projects for 
hard-to-serve groups. 

Emphasis on Different Outcomes 

For PY 88, New York State chose the seven termination-based 
performance standards that had been used previously and the youth 
employability enhancement rate. Before PY 89, the federal standards 
were given equal weight in the incentive award allocation formula. 
Beginning in PY 89, the cost standards were given a zero weight to 
encourage more intensive services, and all other federal standards 
used for the incentive awards were weighted equally. Half of the 
incentive awards were based on performance on these measures. 

The state also established five additional standards for service to 
and outcomes for long-term AFDC recipients and for target groups 
that the SDAs proposed as a local priority. The remaining half of the 
incentive funds were based on performance on these standards: 

■ For long-term AFDC recipients (defined as 24 months or longer 
on the AFDC rolls) and other hard-to-serve adults (based on SDA 
proposals): 

- Level of enrollment relative to incidence in eligible population 
(one-sixth weight). 

- Placement rate (two-sixths weight). 

Average wage at placement (three-sixths weight). 

■ For hard-to-serve youths (based on SDA proposals): 

- Level of enrollment relative to incidence in eligible population 
(one-sixth weight). 

- Positive termination rate (five-sixths weight). 




III-5 



Qualifying for Incentive Awards 



To qualify for an award, an SDA must have a positive average 
performance deviation from the standard across all the federal 
standards, and may not underachieve by 20% or more on any two 
standards. 

Calculation of incentive Awards 

The state has designed primarily a noncompetitive procedure for 
awarding incentives for the federal standards. A separate pool is 
created for each standard, and within that pool, a separate potential 
award is set aside for each SDA, weighted by SDA size. Each SDA 
tries to earn its full potential share. One-half of the maximum award 
is earned by performance that marginally exceeds the standard; three- 
fourths of the maximum award is earned by performance that exceeds 
the standard by at least 5% and less than 10%; the full award is 
earned by performance that exceeds the standard by at least 10%. 

Because not all SDAs earn their full potential share, a second 
distribution is made of 50% of the unearned awards to those SDAs 
that exceeded the standard by 10% or more. This additional 
allocation is distributed in proportion to the SDA's allocation. (The 
remaining unearned awards are used for technical assistance 
activities.) 

The hard-to-serve incentive pool is allocated based on a tiered 
distribution process with a separate reward pool for each of the five 
hard-to-serve measures. The allocation of the hard-to-serve incentive 
funds between the adult and youth measures is based on the relative 
number of adult and youth hard-to-serve participants. 
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Conditions Placed on Incentive Funds 

All incentive awards can be used for any allowable activity. The 
state now permits SDAs to decide whether or not to exclude all 
incentive awards from the performance standards. This policy is 
intended to encourage SDAs to use incentive awards (both funds 
earned for attaining performance standards and hard-to-serve 
incentive funds) for special projects. 

D. PENNSYLVANIA: PROMOTE SERVICE TO THE HARD TO 
SERVE USING A VARIETY OF TOOLS 

Policy Objectives 

Pennsylvania has adopted a number of different strategies to 
promote services to hard-to-serve groups. The performance standards 
system is only one policy tool used in Pennsylvania to promote 
services to groups with multiple employment barriers. 

Another important policy tool is the use of state 8% funds and 
state Title Il-A administrative funds to directly support a series of 
state-initiated demonstration programs (many of which are 
cooperatively established and funded with the state Department of 
Public Welfare and the state Department of Education). Among the 
demonstration programs currently under way are the Youth Urban 
Service Corps, the Single Point of Contact Program (oriented to 
welfare recipients), a demonstration program for at-risk high school 
students, and a demonstration program for pregnant and parenting 
teens. 

Using the incentive allocation formula, Pennsylvania encourages 
services to hard-to-serve groups using two different strategies: 

• It rewards SDAs that exceed their goals for services to three target 
groups: long-term AFDC recipients, dropouts, and teen parents. 
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■ It requires (as mandated by the state legislature) that SDAs use 
their entire incentive awards for services to public assistance 
recipients, at-risk youths aged 14 through 21, and school dropouts. 

Among the key elements of the state's incentive awards design are 
the following features: 

■ The implementation of state standards for the level of service to 
long-term AFDC recipients, dropouts, and teen parents, and the 
award of 20% of the state incentive funds based on these three 
state standards. 

■ An incentive policy that limits competition among SDAs by 
capping rewarded performance at 20% over the standards. 

■ A policy exempting all 6% expenditures and participants from the 
performance standards, starting in PY 88. 

■ A decision to delete the adult and youth cost standards because 
these standards were encouraging too much short-term training. 

Emphasis on Different Outcomes 

In PY 88, Pennsylvania implemented eight federal standards, 
including three follow-up standards and only one youth standard- 
the youth employability enhancement rate. The selected federal 
standards are used to distribute 80% of the incentive awards. A 
composite index is used that nominally weights standards equally. 
For PY 89, the state has decided to replace the cost per entered 
employment standard with the youth positive termination rate 
standard because the cost standard was encouraging too much 
short-term training. In addition, the welfare follow-up 
employment rate is emphasized by establishing it as a core 
standard that must be met to qualify for incentive awards. 

The additional state standards include: service to long-term 
AFDC recipients, service to dropouts, and service to teen parents. 
These standards, used to award a 20% pool of incentive funds, are 
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equally weighted among themselves. In addition, expenditure 
rates are incorporated into the incentive formula. 

Qualifying for Incentive Awards 

The state has defined exceeding a standard as performance at 
or above the lower tolerance level established by the DOL 
adjustment model. To qualify for an incentive award, an SDA 
must exceed seven of the eight standards. A required core 
standard for qualifying for incentive awards is the welfare follow- 
up employment rate. 

To qualify for incentive awards on the optional state standards, an 
SDA must qualify for incentive awards on the federal standards. The 
state has defined the required service level for each hard-to-serve 
group in the state standards to be the incidence of that group in the 
SDA's eligible population. 

Calculation of Incentive Awards 

For PY 88, Pennsylvania used a composite formula to measure 
performance in excess of the federal standards. This formula sums 
the percentage difference between an SDA's performance and its 
standard. The percentage difference was capped at an average 
performance level of approximately 20% above each standard. 

The size of an SDA's incentive award in Pennsylvania is adjusted 
by both SDA size and expenditure rate. The reduction in incentive 
award can be substantial for SDAs with low expenditure rates. No 
adjustment is made if the expenditure rate is at least 85%. However, 
expenditure rates between 75% and 84% result in a 30% reduction in 
the size of the incentive award; expenditure rates between 65% and 
74% result in a 50% reduction in the size of the incentive award, and 
so forth. 

C2 




III-9 



Pennsylvania is considering altering its incentive allocation 
formula so that awards would be distributed without reference to the 
amount by which the standard was exceeded. This is in keeping with 
the state's increasing desire to promote quality services to hard-to- 
serve clients instead of high performance on the standards. 



Conditions Placed on Incentive Funds 

Pennsylvania requires SDAs to use their entire incentive award for 
services to one or more of three target groups: public assistance 
recipients, at-risk youths age 14 to 21, and school dropouts. In 
keeping with this policy, all 6% expenditures and participants are 
automatically exempt from the performance standards. However, 
SDAs must specify what measures of success will be used with 
enrollees in 6% programs. 



E. KENTUCKY: REWARD PERFORMANCE MARGINALLY IN 
EXCESS OF THE STANDARDS AND ENCOURAGE 
PLACEMENT OF INDIVIDUALS FROM SEVEN TARGET 
POPULATIONS 

Policy Objectives 

Kentucky uses its incentive policy to reward performance 
marginally in excess of the standards and to encourage SDAs to 
increase the placement rate of seven target populations identified by 
the State Job Training Coordinating Council. 

Among the key elements of the state's current design strategy are 
the following features: 

■ Deemphasis of the cost standards through elimination of the adult 
cost per entered employment standard. 
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■ An incentive allocation that deemphasizes competition among 
SDAs and distributes the majority of the incentive awards for the 
federal standards for performance marginally in excess of the 
standards. 

■ The provision of state awards for "employment of target 
populations," including school dropouts, AFDC recipients, youths, 
handicapped, offenders, unemployed female single parents, and 
teen parents. 



Emphasis on Different Outcomes 

For PY 88/89, Kentucky implemented 6 adult performance 
standards and 2 youth standards from the 12 federal standards. The 
six adult standards include three termination-based standards and 
three follow-up standards. (This was a transitional strategy to prepare 
for the federal selection of the follow-up standards in PY 90.) The 
adult standard not selected for implementation was the cost per 
entered employment. The two youth standards selected were the 
youth emplo> ability enhancement rate and cost per positive 
termination. Kentucky chose to give each standard a different weight 
in the incentive allocation formula to emphasize priority standards. 

Before PY 88, Kentucky distributed 50% of the incentive awards 
in response to the number of individuals served in each of seven 
targeted population groups. Beginning in PY 88, the state target 
population awards are made in response to the number of individuals 
placed, rather than served , in each of these groups. Thus, the state 
standards currently include an entered employment "count" for each 
of the following groups: 

■ School dropouts 

■ AFDC recipients 

■ Youths 

■ Handicapped 

■ Offenders 
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Unemployed female single parents 
Teen parents. 



In PY 88, the state double-weighted the dropout target group to 
emphasize its concern for services to high school dropouts. 

Qualifying for Incentive Awards 

The SDAs' standards for the federal measures were set at the 
lower boundary of the tolerance range. The definition of "meeting a 
standard" in Kentucky is performing at or above 90% of that level. 
The definition of "exceeding a standard" is performing above the 
lower level of the tolerance range. 

To qualify for an award from the federal standards incentive 
awards pool, an SDA must meet five of the eight federal standards 
and exceed two adult employment standards and the youth 
employability enhancement rate. 

Three of the seven state award pools also have criteria that must 
be met to qualify for targeted-population awards. To qualify for an 
incentive award for placing youth, an SDA must meet its youth 
expenditure rate requirement and, beginning in PY 89, the youth 
target population award will be based on the number of youth who 
both enter-employment and acheive a youth employability 
enhancement. To qualify for incentive awards for placing dropouts 
and AFDC recipients, an SDA must meet its equitable service goal 
for these two groups. 

Calculation of incentive Awards 

Half of incentive funds are used to reward performance on the 
federal standards. These funds are allocated in two rounds. The first 
round is made using a separate pool for each standard according to 
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the weight assigned to that standard. Each SDA has a predetermined 
share equal to each SDA's formula share of 78% funds. If an SDA 
earns its share-by exceeding its standard-the funds are awarded 
during Round 1. If an SDA fails to earn its share on a given standard, 

0 

the funds are set aside to be distributed among high-performing SDAs 
during Round 2. Thus, Round 1 distributions are not competitive. 

The second round of allocation for the federal standards is limited 
to the five SDAs with the highest average performance across all eight 
standards. A summary performance measure is created by summing 
the percentage by which each SDA exceeded each standard, and 
dividing this total by 8. The five SDAs with the highest summary 
scores divide any funds not distributed during Round 1, with the size 
of each SDA's Round 2 share determined by the size of its summary 
score. Although Round 2 distributions of unawarded funds tend to be 
small, they are highly competitive and are not adjusted for SDA size. 

The remaining half of the state's incentive funds are used for 
awards for placement of individuals from the seven target 
populations. Separate pools are established for each target 
population. The pool for dropouts is twice as large as the pool for 
each of the other standards. The size of each SDA's award from each 
target population pool is determined by computing a per-capita award 
for placing an individual from that pool. This competitive formula 
gives substantial incentives to exceed these standards for placing hard- 
to-serve participants. 



Conditions Placed on incentive Funds 

Beginning in PY 88, Kentucky implemented a statewide exemption 
of incentive funds from the federal performance standards. The state 
has no special requirements for SDA expenditure of these funds. 



68 

o IIM3 

ERIC 



F. COLORADO: ENCOURAGING PERFORMANCE WITHIN A 
REALISTIC RANGE 



Policy Objectives 

Colorado's performance standards and incentive policies 
encourage quality placements, discourage SDA competition, and 
reward performance through a continuous award process that limits 
incentives for over-performance. Additionally, the Job Training 
Coordinating Council decided that technical assistance funds should 
be distributed as needed in accordance with approved plans to 
improve performance. Key provisions of Colorado's strategy include 
the following: 

■ Gradual implementation of all postprogram measures and the 
youth employability enhancement rate, coupled with zero 
weighting of the cost standards in the incentive award allocation 
formula, thereby encouraging quality placements. 

■ Creating distinct target pools for each SDA while requiring a 
threshold number of standards to be met for SDAs to be eligible 
for incentive awards. 

■ Establishing a continuous incentive award policy with a cap to 
encourage improved performance without rewarding 
overperformance. 

■ Allocating technical assistance funds based on how well the SDA's 
proposal addresses deficiencies in performance. 

Specific policies implemented by Colorado are outlined 
below. 



Emphasis on Different Outcomes 

Colorado set different performance standards for PY 88 and 
PY 89 to allow a gradual transition to postprogram measures. 
For PY 88, Colorado dropped the youth entered-employment rate 
measure and instituted the adult follow-up employment rate and 
weeks worked measures. In PY 89, the adult wage at placement 
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and welfare entered-employment rate standards were dropped to 
allow full implementation of postprogram measures. 
Additionally, the youth employability enhancement rate was set as 
a state standard for PY 89. This standard is used only to 
determine SDA incentive awards, not for sanctions. 

During both program years, Colorado zero weighted the cost 
standards in the incentive award allocation formula. Colorado 
established separate pools of incentive funds for each standard. 
In PY 89, the youth positive-termination rate standard was given 
the highest weight (25% of available incentives) and the adult 
entered-employment rate the lowest (3%). All other measures 
were weighted between 13% and 15%. This distribution was 
designed to deemphasize cost as a rewardable goal and 
emphasize postprogram employment as the primary SDA goal. 



Qualifying for Incentive Awards 

Colorado has defined a "range of rewardable performance" for 
each standard. The lower bound of this range is set at the lower 
bound of the tolerance range and is capped 10% above the standard. 
Exceeding a standard is defined as performance above the lower 
bound. To be eligible for incentive awards, SDAs must exceed five of 
the eight federal standards. 



Calculation of Incentive Awards 

The state established a continuous, noncompetitive incentive 
award policy with a cap for PY 88/89. Based on an SDA's relative 
share of the state's economically disadvantaged population, a target 
award pool for each SDA is established for each standard. The 
amount of the incentive award received by the SDA is determined by 
the actual performance within the rewardable range. 
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The incentive awards are calculated using a continuous approach. 
The amount of incentives is determined by the proportion of the 
rewardable range achieved. For example, if an SDA's follow-up 
weeks worked standard is 8 weeks, the range of rewardable 
performance extends from 8 to 8.8 weeks. If the SDA's actual 
performance is 8 weeks, the SDA does not receive any funds from this 
target pool. If the SDA's performance is 8.4 weeks, the SDA is 
eligible for half of the available target award funds. Performance at 
or above 8.8 weeks results in a full award. By placing a cap on 
rewardable performance, the state discourages overperformance. 

Conditions Placed on Incentive Funds 

Colorado neither excludes incentive funds from performance 
standards nor requires incentive awards to be used for specified 
groups. 

Annually, the Job Training Coordinating Council sets aside a 
portion of 6% funds to be used for technical assistance. Subsequent 
to the announcement of incentive awards for the prior program year, 
SDAs may submit a proposal for these funds. The proposal should 
include the following elements: 

■ Identification of the performance standards the SDA wishes to 
improve on-this may include both failed standards and standards 
for which the SDA did not receive the maximum award. 

■ An analysis of why the deficiencies occurred. 

■ A description of how the SDA intends to improve performance 

■ A detailed statement outlining the funds required to support 
improved performance. 

This policy distributes technical assistance funds through a process 
that rewards SDAs that identify specific action plans to address 
deficiencies in performance. 
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APPENDIX A 

WEIGHTING PROCEDURE TO TREAT STANDARDS EQUALLY 
IN A COMPOSITE MEASURE OF PERFORMANCE 

As discussed in Chapter II, states that use a composite measure 
of performance that sums the percentage by which each standard is 
exceeded have inadvertently given greater weight to standards that 
are easier to exceed by a wide margin. One way to correct this 
problem is to give standards unequal weights in the composite index 
to undo the inadvertent weighting. 

Standards can be treated dually by using the following weights 
in a composite formula based on percentage above standard: 

W ei gh t 



Adult follow-up employment rate 26% 

Adult follow-up weekly earnings 24 

Welfare follow-up employment rate 15 

Welfare follow-up weekly earnings 19 
Youth employability enhancement rate 6 

Youth entered-employment rate 10 

Total 100 



An SDA performing at the 75th percentile on every standard would 
receive approximately the same credit for each standard when these 
values are used to weight the percentage above standard in a 
composite formula. 

States can adjust these weights to reflect ?tate policy by 
increasing the weights of standards that they want to emphasize and 
reducing the weights for standards that they want to deemphasize. 
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